Exploration

1. Read the data into R/Python
2. Ran summary functions on features
3. Plotted the data
4. Look for patterns and relationships between features
5. Determine what features seperate genders best

Classification

1. Used Scikit-learn in Python
2. Split the data for training/testing (2/3, 1/3)
3. Used gridsearch to identify the best parameters
4. KNN (K-Nearest Neighbors)
5. Decision Tree (DT)
6. Suport Vector Machine (SVM)
7. Logistic Regression (Log R)
8. Observed prediction outcomes. Could do better.
9. Attempt to Improve on initial results
10. KNN: Transform data with PCA
11. Decision Tree: Use multiple trees with Random Forest
12. SVM: Transform data with PCA
13. Log R: Normalized data from 0 to 1 in each feature

Review

• Confusion Matrix
• Overall Accuracy Scores
• Male Accuracy
• Female Accuracy
• ROC & AUC
• Parameter Influence
• Fit & Score Times

• Used untransformed data
• Better then a dumb classifier (50/50)
• Distance weights outperformed Uniform weights
• P: Manhattan Distance produced better CV results (p=1)
• Algorithm: auto attempts to decide the most appropriate algorithm based on values
• Weights: distance weights points by the inverse of their distance. in this case, closer neighbors of a query point will have a greater influence than neighbors which are further away

• algorithm = auto, n_neighbors = 11, p = 1, weights = distance

• Used untransformed data
• MeanFun, sp.ent & IQR account for +90% of feature importance
• Presort: presort the data to speed up the finding of best splits in fitting
• Splitter: The strategy used to choose the split at each node
• Better at identifying males
• Easiest model to interpret (follow the branches)
• Tree

• criterion = gini, max_depth = 21, presort = TRUE, splitter = random

• Modified penalty parameter to acheive better results
• Higher penalties acheived better scores
• Better at classifying males

• C = 48

• Untransformed data
• Best Male accuracy
• Outperformed Log Reg (Normal)
• C: Inverse of regularization strength
• fit_intercept: Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function
• penalty: Used to specify the norm used in the penalization

• C = 0.7000000000000001, fit_intercept = TRUE, penalty = l1

• Best Female accuracy
• Took longer to fit then Decision Tree

• criterion = entropy, max_depth = 9, n_estimators = 15

• Best overall accuracy
• 9 PCA components used
• The fewer the neighbors the better

• algorithm = auto, n_neighbors = 3, p = 1, weights = distance

• Improvement over SVM on untransformed data
• Adjusted penalty parameter C of the error term
• Acheived best performance at much lower penalty parameter levels

• C = 10

• Performed worse then Log Regression on untransformed data
• Decrease in performance due to decrease in Male accuracy
• Slight improvement in Female accuracy compared to first Log R

• C = 0.9000000000000001, fit_intercept = TRUE, penalty = l1